Efficient Incremental Smart Grid Data Analytics

Analytical computations over energy data are gaining popularity thanks to the growing adoption of smart electricity meters. Computations in this context range from seemingly straightforward tasks such as calculating monthly bills based on time-of-use pricing, to elaborate model building for predictions and recommendations in an effort to reduce peak demand. While research in this promising area is progressing steadily, published algorithms and prototypes have largely avoided the important practical question of how to deal efficiently with the incremental nature of energy data, for example per-hour readings produced by smart electricity meters. As a stepping stone towards a comprehensive solution to this problem, we investigate incremental techniques for disaggregating different categories of energy consumption, such as base load versus activity load, from hourly smart meter data using the popular “three-line model” of Birt et al. Our software prototype, called Insparq, exhibits speedups in excess of 2x for data sets up to tens of GB in size, compared to a naive implementation on top of a conventional scalable batch processing framework.